Systems and methods for providing an interactive virtual environment

ABSTRACT

A system includes one or more memory devices storing instructions, and one or more processors configured to execute the instructions to perform steps of a method for providing an interactive virtual environment. The system may receive environmental data, and generate virtual environmental data based on the environmental data. The system may then provide the virtual environmental data to at least a first computing device to create a virtual environment. The system may then receive user input, and modify the virtual environment to display enhanced content in response to the user input.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/664,920, filed Jul. 31, 2017, which claims the benefit of U.S.Provisional Patent Application No. 62/437,706, filed Dec. 22, 2016, bothof which the entire contents and substance are hereby incorporated byreference.

FIELD OF INVENTION

The present disclosure relates to systems and methods for providing aninteractive virtual environment, and more particularly providing aninteractive virtual environment associated with a physical environment.

BACKGROUND

When making travel plans, a traveler may often be faced with difficultdecisions regarding where to go, which sites to see, and whichactivities to engage in, particularly if the traveler is going somewherenew. For example, if a traveler is planning a vacation to a distantdestination (or to one of several distant destinations), the travelermay want to maximize their experience during their limited time atwhichever destination they choose without the benefit of knowing whichtourist sites or activities are available and would be most enjoyable.Because vacations are often expensive and limited in time, decisionsmade in advance about where to go, what to see, and what to do can becrucial to a traveler's ultimate enjoyment of the trip. Travel guidesand websites may provide some information about various tourist sites,attractions, and experiences, but can be inefficiently time-consumingand confusing for the traveler to review, are quickly outdated asavailability, cost, and other details about tourist sites and activitieschange, and are typically not personalized to the traveler.

Accordingly, there is a need for improved systems and methods to helptravelers more efficiently and effectively create their personalizedtravel plans. Embodiments of the present disclosure are directed to thisand other considerations.

SUMMARY

Disclosed embodiments provide systems and methods for providing aninteractive virtual environment.

Consistent with the disclosed embodiments, the system may include one ormore memory devices storing instructions, and one or more processorsconfigured to execute the instructions to perform steps of a method toprovide an interactive virtual environment. The system may execute theinstructions to receive environmental data comprising audio and visualinformation representing a physical environment. The system may thengenerate virtual environmental data based on the environmental data. Thevirtual environmental data may include data associated with a merchanthaving a presence in the physical environment. The system may thenprovide the virtual environmental data to at least a first computingdevice to create a virtual environment. The system may receive userinput data associated with one or more predefined user inputs from thefirst computing device. In response to the user input data, the systemmay modify the virtual environment on the first computing device todisplay enhanced content, where the enhanced content may be content thatis unrepresented in the physical environment.

Consistent with the disclosed embodiments, methods for providing aninteractive virtual environment are also disclosed.

Further features of the disclosed design, and the advantages offeredthereby, are explained in greater detail hereinafter with reference tospecific embodiments illustrated in the accompanying drawings, whereinlike elements are indicated be like reference designators.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made to the accompanying drawings, which are notnecessarily drawn to scale, and which are incorporated into andconstitute a portion of this disclosure, illustrate variousimplementations and aspects of the disclosed technology and, togetherwith the description, serve to explain the principles of the disclosedtechnology. In the drawings:

FIG. 1 is a diagram of an exemplary system that may be used to providean interactive virtual environment;

FIG. 2 is a component diagram of an exemplary service provider terminal;

FIG. 3 is a flowchart of an exemplary method for providing aninteractive virtual environment;

FIG. 4 is a flowchart of an exemplary method for generating virtualenvironmental data based on environmental data;

FIG. 5 is a diagram of exemplary types of virtual environmental data;and

FIG. 6 is a component diagram of an exemplary computing device.

DETAILED DESCRIPTION

Some implementations of the disclosed technology will be described morefully with reference to the accompanying drawings. This disclosedtechnology may, however, be embodied in many different forms and shouldnot be construed as limited to the implementations set forth herein. Thecomponents described hereinafter as making up various elements of thedisclosed technology are intended to be illustrative and notrestrictive. Many suitable components that would perform the same orsimilar functions as components described herein are intended to beembraced within the scope of the disclosed electronic devices andmethods. Such other components not described herein may include, but arenot limited to, for example, components developed after development ofthe disclosed technology.

It is also to be understood that the mention of one or more method stepsdoes not preclude the presence of additional method steps or interveningmethod steps between those steps expressly identified. Similarly, it isalso to be understood that the mention of one or more components in adevice or system does not preclude the presence of additional componentsor intervening components between those components expressly identified.

The disclosed embodiments are directed to systems and methods forproviding an interactive virtual environment that may be used as an aidin travel planning. The system may include one or more memory devicesstoring instructions, and one or more processors configured to executethe instructions to perform steps of a method. Specifically, in someembodiments, the system may provide an interactive virtual environment.To accomplish this, the system may execute the instructions to receiveenvironmental data comprising audio and visual information representinga physical environment. The system may then generate virtualenvironmental data based on the environmental data. The virtualenvironmental data may include data associated with a merchant having apresence in the physical environment. The system may then provide thevirtual environmental data to at least a first computing device tocreate a virtual environment. The system may receive user input dataassociated with one or more predefined user inputs from the firstcomputing device. In response to the user input data, the system maymodify the virtual environment on the first computing device to displayenhanced content, where the enhanced content may be content that isunrepresented in the physical environment.

In another embodiment, a system for providing an interactive virtualenvironment that may be used as an aid in travel planning may include anenvironmental data sensor for obtaining environmental data comprisingaudio and/or visual information representing a physical environment. Thesystem may include a geographic location sensor associated with theenvironmental data sensor for obtaining location data associated withthe physical environmental and an input/output device for receivingmerchant data from a merchant database. The merchant database mayinclude data associated with merchants located at a specific geographiclocation. The system may further include a virtual environment processorthat, based on the location data obtained by the geographic locationsensor, determines merchant data stored in the merchant database that isrelevant to the geographic position of the environmental data sensor,integrates the merchant data with the environmental data to createaugmented environmental data, and generates a signal comprisingaugmented environmental data. The system may also include an augmentedenvironmental data display that receives the signal comprising augmentedenvironmental data and displays said augmented environmental data inaccordance with said signal.

In another aspect, a non-transitory computer-readable medium storinginstructions is disclosed. The instructions, when executed by one ormore processors, may cause a computing device to perform steps of amethod for providing an interactive virtual environment. Specifically,the computing device may receive environmental data comprising audio andvisual information representing a physical environment. The computingdevice may then generate virtual environmental data based on theenvironmental data. The virtual environmental data may include dataassociated with a merchant having a presence in the physicalenvironment. The instructions may then cause the computing device toprovide the virtual environmental data to at least a first computingdevice to create a virtual environment. The computing device may receiveuser input data associated with one or more predefined user inputs fromthe first computing device. In response to the user input data, thecomputing device may modify the virtual environment on the firstcomputing device to display enhanced content, where the enhanced contentmay be content that is unrepresented in the physical environment.

Reference will now be made in detail to exemplary embodiments of thedisclosed technology, examples of which are illustrated in theaccompanying drawings and disclosed herein. Wherever convenient, thesame references numbers will be used throughout the drawings to refer tothe same or like parts.

FIG. 1 is a diagram of an exemplary system that may be configured toperform one or more processes that can provide an interactive virtualenvironment. The components and arrangements shown in FIG. 1 are notintended to limit the disclosed embodiments as the components used toimplement the disclosed processes and features may vary.

In accordance with disclosed embodiments, an interactive virtualenvironment system 100 may include a service provider terminal 110 incommunication with a first computing device 120 and a second computingdevice 130 via network 140. First and second computing devices 120 and130 may include a mobile computing device (e.g., a smart phone, tabletcomputer, smart wearable device, portable laptop computer, voice commanddevice, or other mobile computing device) or a stationary computingdevice (e.g., a stationary desktop or other stationary computer).Computing devices 120 and 130 may belong to or be provided by aconsumer, or may be borrowed, rented, or shared. In some embodiments,computing devices 120 and 130 may be configured to provide one or moreof an augmented reality and a virtual reality experience to one or moreusers, and may be equipped with a display, speakers or other auditorydevices, tactile simulators, and other devices configured to engage theuser's senses to enhance the augmented/virtual reality experience.

In some embodiments, interactive virtual environment system 100 may alsoinclude a merchant database terminal 150 that communicates with otherdevices of system 100 via network 140. Optionally, in some embodiments,interactive virtual environment system 100 may also include a merchantpoint of sale (POS) terminal 160 that communicates with other devices ofsystem 100 via network 140. Additionally, in some embodiments,interactive virtual environment system 100 may also include a financialservice provider terminal 170 that communicates with other devices ofsystem 100 via network 140.

Network 140 may be of any suitable type, including individualconnections via the internet such as cellular or WiFi networks. In someembodiments, network 140 may connect terminals using direct connectionssuch as radio-frequency identification (RFID), near-field communication(NFC), Bluetooth™, low-energy Bluetooth™ (BLE), WiFi™, ZigBee™, ambientbackscatter communications (ABC) protocols, USB, or LAN. Because theinformation transmitted may be personal or confidential, securityconcerns may dictate one or more of these types of connections beencrypted or otherwise secured. In some embodiments, however, theinformation being transmitted may be less personal, and therefore thenetwork connections may be selected for convenience over security.

An exemplary embodiment of service provider terminal 110 is shown inmore detail in FIG. 2. Computing devices 120 and 130, merchant databaseterminal 150, merchant POS terminal 160, and financial service providerterminal 170 may have a similar structure and components that aresimilar to those described with respect to service provider terminal110. As shown, service provider terminal 110 may include a processor210, an input/output (“I/O”) device 220, a memory 230 containing anoperating system (“OS”) 240 and a program 250. For example, serviceprovider terminal 110 may be a single server or may be configured as adistributed computer system including multiple servers or computers thatinteroperate to perform one or more of the processes and functionalitiesassociated with the disclosed embodiments. In some embodiments, theservice provider terminal 110 may further include a peripheralinterface, a transceiver, a mobile network interface in communicationwith the processor 210, a bus configured to facilitate communicationbetween the various components of the service provider terminal 110, anda power source configured to power one or more components of the serviceprovider terminal 110.

A peripheral interface may include the hardware, firmware and/orsoftware that enables communication with various peripheral devices,such as media drives (e.g., magnetic disk, solid state, or optical diskdrives), other processing devices, or any other input source used inconnection with the instant techniques. In some embodiments, aperipheral interface may include a serial port, a parallel port, ageneral purpose input and output (GPIO) port, a game port, a universalserial bus (USB), a micro-USB port, a high definition multimedia (HDMI)port, a video port, an audio port, a Bluetooth™ port, a near-fieldcommunication (NFC) port, another like communication interface, or anycombination thereof.

In some embodiments, a transceiver may be configured to communicate withcompatible devices and ID tags when they are within a predeterminedrange. A transceiver may be compatible with one or more of:radio-frequency identification (RFID), near-field communication (NFC),Bluetooth™, low-energy Bluetooth™ (BLE), WiFi™ZigBee™, ambientbackscatter communications (ABC) protocols or similar technologies.

A mobile network interface may provide access to a cellular network, theInternet, or another wide-area network. In some embodiments, a mobilenetwork interface may include hardware, firmware, and/or software thatallows the processor(s) 210 to communicate with other devices via wiredor wireless networks, whether local or wide area, private or public, asknown in the art. A power source may be configured to provide anappropriate alternating current (AC) or direct current (DC) to powercomponents.

Processor 210 may include one or more of a microprocessor,microcontroller, digital signal processor, co-processor or the like orcombinations thereof capable of executing stored instructions andoperating upon stored data. Memory 230 may include, in someimplementations, one or more suitable types of memory (e.g. such asvolatile or non-volatile memory, random access memory (RAM), read onlymemory (ROM), programmable read-only memory (PROM), erasableprogrammable read-only memory (EPROM), electrically erasableprogrammable read-only memory (EEPROM), magnetic disks, optical disks,floppy disks, hard disks, removable cartridges, flash memory, aredundant array of independent disks (RAID), and the like), for storingfiles including an operating system, application programs (including,for example, a web browser application, a widget or gadget engine, andor other applications, as necessary), executable instructions and data.In one embodiment, the processing techniques described herein areimplemented as a combination of executable instructions and data withinthe memory 230.

Processor 210 may be one or more known processing devices, such as amicroprocessor from the Pentium™ family manufactured by Intel™ or theTurion™ family manufactured by AMD™. Processor 210 may constitute asingle core or multiple core processor that executes parallel processessimultaneously. For example, processor 210 may be a single coreprocessor that is configured with virtual processing technologies. Incertain embodiments, processor 210 may use logical processors tosimultaneously execute and control multiple processes. Processor 210 mayimplement virtual machine technologies, or other similar knowntechnologies to provide the ability to execute, control, run,manipulate, store, etc. multiple software processes, applications,programs, etc. One of ordinary skill in the art would understand thatother types of processor arrangements could be implemented that providefor the capabilities disclosed herein.

Service provider terminal 110 may include one or more storage devicesconfigured to store information used by processor 210 (or othercomponents) to perform certain functions related to the disclosedembodiments. In one example, service provider terminal 110 may includememory 230 that includes instructions to enable processor 210 to executeone or more applications, such as server applications, networkcommunication processes, and any other type of application or softwareknown to be available on computer systems. Alternatively, theinstructions, application programs, etc. may be stored in an externalstorage or available from a memory over a network. The one or morestorage devices may be a volatile or non-volatile, magnetic,semiconductor, tape, optical, removable, non-removable, or other type ofstorage device or tangible computer-readable medium.

In one embodiment, service provider terminal 110 may include memory 230that includes instructions that, when executed by processor 210, performone or more processes consistent with the functionalities disclosedherein. Methods, systems, and articles of manufacture consistent withdisclosed embodiments are not limited to separate programs or computersconfigured to perform dedicated tasks. For example, service providerterminal 110 may include memory 230 that may include one or moreprograms 250 to perform one or more functions of the disclosedembodiments. Moreover, processor 210 may execute one or more programs250 located remotely from system 100. For example, system 100 may accessone or more remote programs 250, that, when executed, perform functionsrelated to disclosed embodiments.

Memory 230 may include one or more memory devices that store data andinstructions used to perform one or more features of the disclosedembodiments. Memory 230 may also include any combination of one or moredatabases controlled by memory controller devices (e.g., server(s),etc.) or software, such as document management systems, Microsoft™ SQLdatabases, SharePoint™ databases, Oracle™ databases, Sybase™ databases,or other relational or non-relational databases. Memory 230 may includesoftware components that, when executed by processor 210, perform one ormore processes consistent with the disclosed embodiments. In someembodiments, memory 230 may include a merchant information database 260,an environment database 270, and a virtual environmental database 280for storing related data to enable service provider terminal 110 toperform one or more of the processes and functionalities associated withthe disclosed embodiments. Merchant information database 260 may includestored data relating to product price (e.g., price of a hotel room,excursion, tour, massage, etc.), product inventory (e.g., availabilityof oceanfront hotel rooms, available time slots for an excursion, etc.),related products, and available discounts associated with one or moreproducts associated with one or more merchants. Although this disclosurerefers to “products” throughout, it should be understood that the termproduct or products may refer to goods, services, items, excursions,tours, or other activities or experiences that are offered for sale atthe physical location or destination. Additionally, the terms “product,”“service,” and “excursion” may be used interchangeably throughout thedisclosure. Environment database 270 may include stored data relating toa physical environment that is captured by one or more devices (e.g.,merchant POS terminal 160 or computing devices 120 and 130) of system100. Virtual environment database 280 may include stored data relatingto a virtual environment that is provided to and can be updated basedupon interactions with one or more devices (e.g., computing devices 120and 130) of system 100.

Service provider terminal 110 may also be communicatively connected toone or more memory devices (e.g., databases (not shown)) locally orthrough a network. The remote memory devices may be configured to storeinformation and may be accessed and/or managed by service providerterminal 110. By way of example, the remote memory devices may bedocument management systems, Microsoft™ SQL database, SharePoint™databases, Oracle™ databases, Sybase™ databases, or other relational ornon-relational databases. Systems and methods consistent with disclosedembodiments, however, are not limited to separate databases or even tothe use of a database.

Service provider terminal 110 may also include one or more I/O devices220 that may comprise one or more interfaces for receiving signals orinput from devices and providing signals or output to one or moredevices that allow data to be received and/or transmitted by serviceprovider terminal 110. For example, service provider terminal 110 mayinclude interface components, which may provide interfaces to one ormore input devices, such as one or more keyboards, mouse devices, touchscreens, track pads, trackballs, scroll wheels, digital cameras,microphones, sensors, and the like, that enable service providerterminal 110 to receive data from one or more users (such as, forexample, via computing devices 120 and 130).

In exemplary embodiments of the disclosed technology, service providerterminal 110 may include any number of hardware and/or softwareapplications that are executed to facilitate any of the operations. Theone or more I/O interfaces may be utilized to receive or collect dataand/or user instructions from a wide variety of input devices. Receiveddata may be processed by one or more computer processors as desired invarious implementations of the disclosed technology and/or stored in oneor more memory devices.

While service provider terminal 110 has been described as one form forimplementing the techniques described herein, those having ordinaryskill in the art will appreciate that other, functionally equivalenttechniques may be employed. For example, as known in the art, some orall of the functionality implemented via executable instructions mayalso be implemented using firmware and/or hardware devices such asapplication specific integrated circuits (ASICs), programmable logicarrays, state machines, etc. Furthermore, other implementations of theterminal 110 may include a greater or lesser number of components thanthose illustrated.

Merchant database terminal 150 may have one or more processors 152 andone or more merchant databases 154, which may be any suitable repositoryof merchant data. Merchant database terminal 150 may be located at thePOS location, off-site at another merchant location, or at a third partylocation. Information stored in merchant database terminal 150 may beaccessed (e.g., retrieved, updated, and added to) via network 140 by oneor more devices (e.g., service provider terminal 110) of system 100.Merchant database 154 may store information relating to products andservices offered by merchants such as pricing, quantity, availability,discounts, reviews, and any other such generally available informationthat a consumer may utilize in making a purchasing decision. In someembodiments, merchant database 154 may also include location informationassociated with products and services that identifies the location(s)that a particular product or service is available for purchase. In someembodiments, the location information may include an identification of aparticular store, terminal, or kiosk that the product or service may bepurchased from.

Merchant POS terminal 160 may have one or more POS devices 162, 164, 166that communicate with one or more devices (e.g., computing devices 120and 130) of system 100 via network 140. In some embodiments, POS devices162, 164, 166 may be associated with one or more products or items at aPOS location, and may serve as beacons to identify the associatedproducts or items and related information to one or more devices ofsystem 100.

Financial service provider terminal 170 may have one or more processors172 and one or more financial service provider databases 174, which maybe any suitable repository of merchant data. Financial service providerterminal 170 may be located at the POS location, off-site at anotherfinancial service provider location, or at a third party location.Information stored in financial service provider database 174 may berequested and sent to one or more devices (e.g., service providerterminal 110, merchant POS terminal 160) of system 100 via network 140.

FIG. 6 shows an exemplary embodiment of first computing device 120. Asshown, first computing device 120 may include input/output (“I/O”)device 220, memory 230 containing operating system (“OS”) 240 andprogram 250 and all associated components as described above withrespect to service provider terminal 110. First computing device 120 mayalso have one or more processors, including a virtual environmentprocessor 602 for generating signals representative of augmented and/orvirtual environments, a geographic location sensor (“GLS”) 604 fordetermining the geographic location of first computing device 120 and/oran associated environmental data sensor 608, an augmented environmentaldisplay 606 for displaying augmented environmental data, and anenvironmental data (“ED”) sensor 608 for obtaining environmental dataincluding audio and/or visual information representing a physicalenvironment. In some embodiments, environmental data sensor 608 mayinclude a microphone or a digital camera. According to some embodiments,virtual environment processor 602 may include all of the features andfunctions of processor 210 described above. Further, virtual environmentprocessor 602 may be configured to determine merchant data stored in amerchant database that is relevant to the geographic position of theenvironmental data sensor based on location data obtained by thegeographic location sensor 604. For example, upon determining thelocation of the environmental data sensor 608, virtual environmentprocessor 602 may determine and obtain merchant data from the merchantdatabase 154 that is associated with products and services that arewithin a predetermined distance from the environmental data sensor 608.Virtual environment processor 602 may integrate the merchant data withthe environmental data to create an augmented environmental data, whichmay be used by the virtual environment processor 602 to generate asignal comprising augmented environmental data (an “augmentedenvironment data signal”) or a virtual environmental data signal. Insome embodiments, an augmented environment data signal may be used tocreate an augmented reality display of the physical environment and avirtual environment data signal may be used to create a virtual realitydisplay of the physical environment. The merchant data associated withthe physical environment may be incorporated into either of theaugmented reality display or the virtual reality display. According tosome embodiments, the virtual environment processor 602 may transmit avirtual environmental data signal to a computing device that maygenerate a virtual environment display based on the virtualenvironmental data signal. Although FIG. 6 is described with respect toa first computing device 120, it is contemplated that in someembodiments, second computing device 130 may include some or all of theelements disclosed with respect to FIG. 6.

FIG. 3 shows a flowchart of method 300 for providing an interactivevirtual environment. Method 300 may be performed by service providerterminal 110 using processor 210 to execute memory 230. In someembodiments, steps of method 300 may be delegated to other elements insystem 100, such as the computing devices 120 and 130 and/or merchantdatabase 150. Following method 300, one or more users may generate,share, and modify virtually augmented environments.

In block 310, service provider terminal 110 may receive environmentaldata comprising audio and visual information. In some embodiments, theenvironmental data may be associated with a physical environment, suchas the physical environment at a travel destination (e.g., an oceanfronthotel room, an ancient ruins tour, a zipline excursion, a massage at thehotel spa, etc.).

After receiving the environmental data, at step 320, service providerterminal 110 may generate virtual environmental data based on theenvironmental data. The virtual environmental data may provide a virtualrepresentation of the physical environment associated with theenvironmental data. The virtual environmental data may include one ormore of product price information, product inventory information,related products, available discounts, and aesthetic visual effects.Product price information may include, for example, the price of aproduct, the price of an excursion or tour, the price of an activity,such as a massage or an amusement park ride, the price of a hotel room,or the price of any other product or service. Product inventoryinformation may include, for example, the number of products availablefor purchase, available hotel rooms or hotels rooms of a certain tier,restaurant reservation times available for booking, the number ofavailable seats or spots (and times) on an excursion or tour, or anyother such information relating to the availability of a product,service, or experience. Related products may include, for example,alternative products or services that may be available, such as rooms ina different hotel or hotel rooms of another tier, dinner reservations ata different restaurant, tours of another location or from anothervendor, and the like. The virtual environmental data may include dataassociated with a merchant having a presence in the physicalenvironment. For example, the merchant may have a presence in thephysical environment by offering one or more products or services forsale in the physical environment or by advertising products/services inthe physical environment.

As shown in FIG. 4, according to some embodiments, the step 320 ofgenerating virtual environmental data based on the environmental datamay include one or more sub-steps. For example, in block 410 the serviceprovider terminal 110 may identify one or more products in theenvironmental data. For example, service provider terminal 110 mayidentify one or more products (e.g., hotel room, excursion, massage,etc.) in the audio or visual information by recognizing shapes, barcodes, QR codes, IR beacons or Bluetooth™ beacons (e.g., POS devices162, 164, 166) located in an environment, or the like. At step 430,service provider terminal 110 may receive product information (such asproduct/service price, availability, and related products/services asdescribed above) from the merchant, from for example, merchant databaseterminal 140. The product information may be directed to one or moreidentified products and services. After receiving product information atstep 420, service provider terminal 110 may then incorporate at least aportion of the received product information into the virtualenvironmental data.

Referring back to FIG. 3, after service provider terminal 110 generatesvirtual environmental data (at block 320), at step 330 service providerterminal 110 may provide the virtual environmental data to at least afirst computing device to create a virtual environment. For example,service provider terminal 110 may provide the virtual environmental datato first computing device 120 and/or to second computing device 130. Insome embodiments, the virtual environment data provided to one or moreof computing devices 120 and 130 may include some or all of theenvironmental data received by the service provider terminal 110.

At step 340, service provider terminal 110 may receive, from firstcomputing device 120, user input data associated with one or morepredefined user inputs. The user input data associated with one or morepredefined user inputs may include a selection of a product displayed inthe virtual environment. For example, a user may select a productdisplayed in the virtual environment in order to purchase or viewinformation about the product. The user input data associated with oneor more predefined user inputs may include a selection of a virtualexperience associated with a product offered for sale at the physicalenvironment. A virtual experience may include one or more of a virtualtour, a virtual excursion, and a virtual activity. Virtual experiencesmay correspond to actual experiences in the physical environment. Forexample, a virtual tour may provide a user with a virtual representationof what the actual tour would look like in real life at the traveldestination. A virtual activity may provide a user with the virtualexperience of the activity, for example, a user may virtually experiencea roller coaster or a scuba dive of a particular location to determineif they are interested in experiencing the real thing (and whether, tothem, it is worth the cost and time).

At step 350, service provider terminal 110 may modify the virtualenvironment on first computing device 120 to display enhanced contentbased on the user input data. The enhanced content may be content thatis unrepresented in the physical environment, such as, for example,product/service information and/or virtual experiences, as describedabove. In some embodiments, modifying the virtual environment on firstcomputing device 120 includes outputting product/service information fordisplay on first computing device 120. In other embodiments, modifyingthe virtual environment on first computing device 120 includesoutputting the selected virtual experience for display on firstcomputing device 120.

According to some embodiments, the method 300 may further include anadditional step whereby the service provider terminal 110 may receive,from the first computing device (e.g., computing device 120), purchaseuser input associated with one or more predefined purchase user inputs,and transmit a purchase request to purchase the selected product orservice offered for sale at the physical environment based on thepurchase input.

In some embodiments, the step of providing the virtual environmentaldata to at least first computing device 120 may further includeproviding the virtual environmental data to the first computing device120 and to a second computing device 130 to create a shared visualenvironment. For example, a shared virtual environment may be a virtualenvironment that is the same on the first computing device 120 and thesecond computing device 130 so that users of both devices can sharetheir experience together. Further, in some embodiments, the step 350 ofmodifying the virtual environment may further include modifying theshared virtual environment on the first 120 and second 130 computingdevices to display the enhanced content based on the user input data.For example, if a user selects (e.g., via first computing device 120) aparticular virtual excursion, the virtual excursion may be displayed onboth the first and second computing devices 120 and 130 simultaneously.Accordingly, system 100 may enable the user of the first computingdevice 120 and the user of the second computing device 130 tosimultaneously explore and interact with the same virtual environment.For example, by utilizing the disclosed system, two friends who aretrying to plan a vacation together despite being physically located indifferent places may nonetheless jointly virtually experience aparticular destination or activity together, all before travelling tothat destination or before experience an excursion, tour, or otherproduct, so that they may make collective decisions about where to goand what activities to engage in.

In some embodiments, the step 310 of receiving the environmental datamay further include receiving the environmental data from secondcomputing device 130, and the second computing device 130 may be locatedat the physical environment at the travel destination or point ofinterest. In such embodiments, the step 330 of providing the virtualenvironment data to at least first computing device 120 may furtherinclude providing the virtual environmental data to first computingdevice 120 to create the virtual environment and to second computingdevice 130 to create an augmented environment in combination with thephysical environment. Further, the step 350 of modifying the virtualenvironment may further include modifying the virtual environment on thefirst computing device 120 and the augmented environment on the secondcomputing device 130 to display the enhanced content based on the userinput data. Accordingly, in some embodiments, system 100 may enable theuser of first computing device 120 to experience a virtual environmentthat is representative of a physical environment that the user of secondcomputing device 130 is physically present in, while presenting thesecond user with a corresponding augmented environment via secondcomputing device 130. For example, by utilizing the disclosed system(e.g., system 100), two friends who are trying to plan a vacationtogether despite being physically located in different places maynonetheless jointly experience a particular destination or activitytogether, wherein the first friend experiences it virtually and thesecond friend experiences it via augmented reality, so that they maymake collective decisions about where to go and what activities toengage in, thereby adding a social experience to the experience offeredby the destination or point of interest.

FIG. 5 shows a diagram that represents an embodiment of exemplary typesof virtual environmental data 500. For example, virtual environmentaldata 500 may include product price information 510, product inventoryinformation 520, related products 530, available discounts 540, orparticular visual effects 550. Product price information 510 mayinclude, for example, the price of a product, the price of an excursionor tour, the price of an activity, such as a massage or an amusementpark ride, the price of a hotel room, or the price of any other productor service. Product inventory information 520 may include, for example,the number of products available for purchase, available hotel rooms orhotels rooms of a certain tier, restaurant reservation times availablefor booking, the number of available seats or spots (and times) on anexcursion or tour, or any other such information relating to theavailability of a product, service, or experience. Related products 530may include, for example, alternative products that may be available,such as rooms in a different hotel or hotel rooms of another tier,dinner reservations at a different restaurant, tours of another locationor from another vendor, and the like.

As used in this application, the terms “component,” “module,” “system”and the like are intended to include a computer-related entity, such asbut not limited to hardware, firmware, a combination of hardware andsoftware, software, or software in execution. For example, a componentmay be, but is not limited to being, a process running on a processor,an object, an executable, a thread of execution, a program, and/or acomputer. By way of illustration, both an application running on acomputing device and the computing device can be a component. One ormore components can reside within a process and/or thread of executionand a component may be localized on one computer and/or distributedbetween two or more computers. In addition, these components can executefrom various computer readable media having various data structuresstored thereon. The components may communicate by way of local and/orremote processes such as in accordance with a signal having one or moredata packets, such as data from one component interacting with anothercomponent in a local system, distributed system, and/or across a networksuch as the Internet with other systems by way of the signal.

Certain embodiments and implementations of the disclosed technology aredescribed above with reference to block and flow diagrams of systems andmethods and/or computer program products according to exampleembodiments or implementations of the disclosed technology. It will beunderstood that one or more blocks of the block diagrams and flowdiagrams, and combinations of blocks in the block diagrams and flowdiagrams, respectively, can be implemented by computer-executableprogram instructions. Likewise, some blocks of the block diagrams andflow diagrams may not necessarily need to be performed in the orderpresented, may be repeated, or may not necessarily need to be performedat all, according to some embodiments or implementations of thedisclosed technology.

These computer-executable program instructions may be loaded onto ageneral-purpose computer, a special-purpose computer, a processor, orother programmable data processing apparatus to produce a particularmachine, such that the instructions that execute on the computer,processor, or other programmable data processing apparatus create meansfor implementing one or more functions specified in the flow diagramblock or blocks. These computer program instructions may also be storedin a computer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meansthat implement one or more functions specified in the flow diagram blockor blocks.

As an example, embodiments or implementations of the disclosedtechnology may provide for a computer program product, including acomputer-usable medium having a computer-readable program code orprogram instructions embodied therein, said computer-readable programcode adapted to be executed to implement one or more functions specifiedin the flow diagram block or blocks. Likewise, the computer programinstructions may be loaded onto a computer or other programmable dataprocessing apparatus to cause a series of operational elements or stepsto be performed on the computer or other programmable apparatus toproduce a computer-implemented process such that the instructions thatexecute on the computer or other programmable apparatus provide elementsor steps for implementing the functions specified in the flow diagramblock or blocks.

Accordingly, blocks of the block diagrams and flow diagrams supportcombinations of means for performing the specified functions,combinations of elements or steps for performing the specifiedfunctions, and program instruction means for performing the specifiedfunctions. It will also be understood that each block of the blockdiagrams and flow diagrams, and combinations of blocks in the blockdiagrams and flow diagrams, can be implemented by special-purpose,hardware-based computer systems that perform the specified functions,elements or steps, or combinations of special-purpose hardware andcomputer instructions.

Certain implementations of the disclosed technology are described abovewith reference to mobile computing devices. Those skilled in the artrecognize that there are several categories of mobile devices, generallyknown as portable computing devices that can run on batteries but arenot usually classified as laptops. For example, mobile devices caninclude, but are not limited to portable computers, tablet PCs, internettablets, PDAs, ultra mobile PCs (UMPCs), wearable devices, and smartphones. Additionally, implementations of the disclosed technology can beutilized with internet of things (IoT) devices, smart televisions andmedia devices, appliances, automobiles, toys, and voice command devices,along with peripherals that interface with these devices.

In this description, numerous specific details have been set forth. Itis to be understood, however, that implementations of the disclosedtechnology may be practiced without these specific details. In otherinstances, well-known methods, structures and techniques have not beenshown in detail in order not to obscure an understanding of thisdescription. References to “one embodiment,” “an embodiment,” “someembodiments,” “example embodiment,” “various embodiments,” “oneimplementation,” “an implementation,” “example implementation,” “variousimplementations,” “some implementations,” etc., indicate that theimplementation(s) of the disclosed technology so described may include aparticular feature, structure, or characteristic, but not everyimplementation necessarily includes the particular feature, structure,or characteristic. Further, repeated use of the phrase “in oneimplementation” does not necessarily refer to the same implementation,although it may.

Throughout the specification and the claims, the following terms take atleast the meanings explicitly associated herein, unless the contextclearly dictates otherwise. The term “connected” means that onefunction, feature, structure, or characteristic is directly joined to orin communication with another function, feature, structure, orcharacteristic. The term “coupled” means that one function, feature,structure, or characteristic is directly or indirectly joined to or incommunication with another function, feature, structure, orcharacteristic. The term “or” is intended to mean an inclusive “or.”Further, the terms “a,” “an,” and “the” are intended to mean one or moreunless specified otherwise or clear from the context to be directed to asingular form. By “comprising” or “containing” or “including” is meantthat at least the named element, or method step is present in article ormethod, but does not exclude the presence of other elements or methodsteps, even if the other such elements or method steps have the samefunction as what is named.

While certain embodiments of this disclosure have been described inconnection with what is presently considered to be the most practicaland various embodiments, it is to be understood that this disclosure isnot to be limited to the disclosed embodiments, but on the contrary, isintended to cover various modifications and equivalent arrangementsincluded within the scope of the appended claims. Although specificterms are employed herein, they are used in a generic and descriptivesense only and not for purposes of limitation.

This written description uses examples to disclose certain embodimentsof the technology and also to enable any person skilled in the art topractice certain embodiments of this technology, including making andusing any apparatuses or systems and performing any incorporatedmethods. The patentable scope of certain embodiments of the technologyis defined in the claims, and may include other examples that occur tothose skilled in the art. Such other examples are intended to be withinthe scope of the claims if they have structural elements that do notdiffer from the literal language of the claims, or if they includeequivalent structural elements with insubstantial differences from theliteral language of the claims.

Exemplary Use Cases

The following exemplary use case describes one example of a typical userflow pattern. It is intended solely for explanatory purposes and not inlimitation. A user in New York may want to plan a vacation to London.The user may use their computing device (e.g., first computing device120) to view and experience an interactive virtual representation ofLondon (e.g., generated based on data provided by service providerterminal 110) that may allow the user to virtually walk around the cityto get a sense of what the attractions look like in real life and/or tovirtually experience some of those attractions. For example, the usermay experience and explore virtual representations of popular publicspaces located in London, such as Piccadilly Circus, Trafalgar Square,Buckingham Palace, etc., or may take a virtual tour of such spaces.Based on these virtual explorations/experiences, the user may gain anappreciation for what a real experience at these locations may consistof, and therefore may be positioned to make more informed decisionsabout which places the user wants to visit during their limited timestay in London or to enjoy the trip as much as possible while keepingwithin a trip budget. Further, the system (e.g., system 100) may display(e.g., via first computing device 120) enhanced content as a part of thevirtual environment, such as information about products, services, andactivities (e.g., received by service provider terminal 110 frommerchant database terminal 140). For example, while the user isexperiencing virtually walking around London, they may see, for example,a hotel, and be presented with information about the availability andpricing of rooms. Or the user may see a bicycle rental merchant and bepresented with the price of renting a bicycle or the times/availabilityof a bicycle tour of a certain area. The user may also experiencevarious excursions, tours, or other activities or experiences. Forexample, the user may be curious about what riding the London Eye ferriswheel would be like, and the system (e.g., system 100) may simulate theexperience of riding the London Eye in the virtual environment so theuser can appreciate what it would be like in real life. Further, thesystem (e.g., system 100) may also enable a plurality of users toexperience a shared virtual environment together simultaneously (e.g.,by service provider terminal 110 providing the virtual environment datato first computing device 120 and to second computing device 130).Accordingly, users in different locations may nonetheless virtuallyexperience a destination at the same time together in order to plan atrip together.

Another use case would be for simultaneously providing a virtualenvironment to a first user (e.g., via a first computing device 120) andan augmented environment to a second user (e.g., via a second computingdevice 130). For example, the first user may again be interested inplanning a trip to London, but in this case, a second user may alreadybe physically present in London. In this scenario, the virtualenvironment experienced by the first user may be augmented withenvironmental data obtained by the computing device of the second user,such that the first user's virtual environment may be based on what thesecond user is seeing and hearing. Furthermore, both users may viewenhanced content, such as product/service information andexcursion/activity information as described above. For example, thesystem (e.g., system 100) may determine the location of the second user(e.g., via a geographic location sensor of second computing device 130)and access merchant data (e.g., by via merchant database terminal 150)relating to products and services within 500 feet of the second user.Accordingly, the first and second user can simultaneously view prices ofproducts, services, hotels, restaurants, activities, excursions, and thelike that they may be interested in booking together. In this way, thesystem (e.g., system 100) may facilitate a local user assisting a remoteuser in planning a trip to the local vicinity.

Other exemplary use cases involve interactions between users each havingaugmented reality devices as described in the systems and methodsherein. Additionally, other exemplary use cases involve interactionsbetween users each having virtual reality devices as described in thesystems and methods herein.

Certain implementations of the disclosed technology are described abovewith reference to block and flow diagrams of systems and methods and/orcomputer program products according to example implementations of thedisclosed technology. It will be understood that one or more blocks ofthe block diagrams and flow diagrams, and combinations of blocks in theblock diagrams and flow diagrams, respectively, can be implemented bycomputer-executable program instructions. Likewise, some blocks of theblock diagrams and flow diagrams may not necessarily need to beperformed in the order presented, may be repeated, or may notnecessarily need to be performed at all, according to someimplementations of the disclosed technology.

These computer-executable program instructions may be loaded onto ageneral-purpose computer, a special-purpose computer, a processor, orother programmable data processing apparatus to produce a particularmachine, such that the instructions that execute on the computer,processor, or other programmable data processing apparatus create meansfor implementing one or more functions specified in the flow diagramblock or blocks. These computer program instructions may also be storedin a computer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meansthat implement one or more functions specified in the flow diagram blockor blocks. As an example, implementations of the disclosed technologymay provide for a computer program product, including a computer-usablemedium having a computer-readable program code or program instructionsembodied therein, said computer-readable program code adapted to beexecuted to implement one or more functions specified in the flowdiagram block or blocks. Likewise, the computer program instructions maybe loaded onto a computer or other programmable data processingapparatus to cause a series of operational elements or steps to beperformed on the computer or other programmable apparatus to produce acomputer-implemented process such that the instructions that execute onthe computer or other programmable apparatus provide elements or stepsfor implementing the functions specified in the flow diagram block orblocks.

As used herein, unless otherwise specified the use of the ordinaladjectives “first,” “second,” “third,” etc., to describe a commonobject, merely indicate that different instances of like objects arebeing referred to, and are not intended to imply that the objects sodescribed must be in a given sequence, either temporally, spatially, inranking, or in any other manner.

1. A system for providing an interactive virtual environment,comprising: one or more processors; and a memory in communication withthe one or more processors and storing instructions that, when executedby the one or more processors, cause the system to: receiveenvironmental data comprising audio and visual information representinga physical environment, the environmental data being obtained from atleast one environmental data sensor associated with a second computingdevice; generate virtual environmental data based on the environmentaldata, the virtual environmental data comprising data associated with amerchant having a presence in the physical environment; transmit thevirtual environmental data to at least a first computing device to allowdisplay of a virtual environment by the first computing device and tothe second computing device to allow display of an augmented environmenton the second computing device such that users of the first computingdevice and the second computing device have a shared virtual andaugmented environment, respectively.
 2. The system of claim 1, whereinthe one or more processors are further configured to execute theinstructions to receive, from the first computing device, user inputdata associated with one or more predefined user inputs, the user inputdata comprising a selection of the product displayed in the virtualenvironment, wherein the selection of the product comprises a selectionof a virtual experience associated with an activity or experienceoffered for sale at the physical environment, the virtual experiencecomprising one or more of a virtual tour, a virtual excursion, and avirtual activity.
 3. The system of claim 2, wherein the one or moreprocessors are further configured to execute the instructions to modifythe virtual environmental data transmitted to the first computing deviceand the second computing device to include enhanced content based on theuser input data such that the enhanced content is displayed by the firstcomputing device and the second computing device in the shared virtualand augmented environments, respectively, wherein the enhanced contentis unrepresented in the physical environment and modifying the virtualenvironment comprises outputting the selected virtual experience fordisplay by the first computing device and the second computing device.4. The system of claim 3, wherein generating the virtual environmentaldata comprises: identifying a product in the environmental data;receiving product information associated with the product from amerchant database; and incorporating at least a portion of the productinformation into the virtual environmental data.
 5. The system of claim4, wherein the virtual environmental data comprises one or more of:product price information, product inventory information, relatedproducts, available discounts, and aesthetic visual effects.
 6. Thesystem of claim 4, wherein the merchant has the presence in the physicalenvironment by offering one or more products and services for sale inthe physical environment, and the received environmental data comprisesproduct information associated with the one or more products offered forsale in the physical environment.
 7. The system of claim 4, wherein theone or more processors are further configured to execute theinstructions to receive, from the first computing device, purchase userinput associated with one or more predefined purchase user inputs, andtransmitting a purchase request to purchase the product offered for saleat the physical environment that is associated with the selected virtualexperience based on the purchase user input.
 8. A method for providingan interactive virtual environment, comprising: receiving, environmentaldata comprising audio and visual information representing a physicalenvironment, the environmental data being obtained from at least oneenvironmental data sensor associated with a second computing device;generating virtual environmental data based on the environmental data,the virtual environmental data comprising data associated with amerchant having a presence in the physical environment; transmitting thevirtual environmental data to at least a first computing device to allowfor display of a virtual environment by the first computing device andto the second computing device to allow display of an augmentedenvironment by the second computing device such that users of the firstcomputing device and the second computing device have a shared virtualand augmented environment, respectively.
 9. The method of claim 8,further comprising: receiving, from the first computing device, userinput data associated with one or more predefined user inputs, the userinput data comprising a selection of the product displayed in thevirtual environment, wherein the selection of the virtual productcomprises a selection of a virtual experience associated with anactivity or experience offered for sale at the physical environment, thevirtual experience comprising one or more of a virtual tour, a virtualexcursion, and a virtual activity.
 10. The method of claim 9, furthercomprising: modifying the virtual environmental data transmitted to thefirst computing device and to the second computing device to includeenhanced content based on the user input data such that the enhancedcontent is displayed by the first computing device and the secondcomputing device in the shared virtual and augmented environments,respectively, wherein the enhanced content is unrepresented in thephysical environment and modifying the virtual environment comprisesoutputting the selected virtual experience for display on the firstcomputing device and the second computing device.
 11. The method ofclaim 10, wherein generating the virtual environmental data comprises:identifying a product in the environmental data; receiving productinformation associated with the product from a merchant database; andincorporating at least a portion of the product information into thevirtual environmental data.
 12. The method of claim 11, wherein thevirtual environmental data comprises at least one element of productprice information, product inventory information, related products,available discounts, and aesthetic visual effects.
 13. The method ofclaim 11, wherein the merchant has the presence in the physicalenvironment by offering one or more products for sale in the physicalenvironment, and the received environmental data comprises productinformation associated with the one or more products offered for sale inthe physical environment.
 14. The method of claim 11, furthercomprising: receiving, from the first computing device, purchase userinput associated with one or more predefined purchase user inputs; andtransmitting a purchase request to purchase the selected product offeredfor sale at the physical environment based on the purchase user input.15. A system for providing an interactive virtual environment,comprising: one or more processors; and a memory in communication withthe one or more processors and storing instructions that, when executedby the one or more processors, cause the system to: receiveenvironmental data comprising audio and visual information representinga physical environment, the environmental data being obtained from atleast one environmental data sensor associated with a first computingdevice; generate virtual environmental data based on the environmentaldata, the virtual environmental data comprising data associated with amerchant having a presence in the physical environment; transmit thevirtual environmental data to the first computing device to allowdisplay of a virtual environment by the first computing device and to asecond computing device to allow display of an augmented environment onthe second computing device such that users of the first computingdevice and the second computing device have a shared virtual andaugmented environment, respectively; receive, from the first computingdevice, user input data associated with one or more predefined userinputs, the user input data comprising a selection of the productdisplayed in the virtual environment; and modify the virtualenvironmental data transmitted to the first computing device and thesecond computing device to include enhanced content based on the userinput data such that the enhanced content is displayed by the firstcomputing device and the second computing device in the shared virtualand augmented environments, respectively.
 16. The system of claim 15,wherein the selection of the product comprises a selection of a virtualexperience associated with an activity or experience offered for sale atthe physical environment, the virtual experience comprising one or moreof a virtual tour, a virtual excursion, and a virtual activity
 17. Thesystem of claim 16, wherein the enhanced content is unrepresented in thephysical environment and modifying the virtual environment comprisesoutputting the selected virtual experience for display by the firstcomputing device and the second computing device.
 18. The system ofclaim 15, wherein generating the virtual environmental data comprises:identifying a product in the environmental data; receiving productinformation associated with the product from a merchant database; andincorporating at least a portion of the product information into thevirtual environmental data.
 19. The system of claim 15, wherein thevirtual environmental data comprises at least one element of productprice information, product inventory information, related products,available discounts, and aesthetic visual effects.
 20. The system ofclaim 19, wherein the virtual environmental data comprises the productprice information comprising at least one price of a price of a product,a price of an excursion, a price of an activity, and a price of a hotelroom.